Finite-size scaling in random /^-satisfiability problems 
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We provide a comprehensive view of various phase transitions in random iiT-satisfiability problems 
solved by stochastic-local-search algorithms. In particular, we focus on the finite-size scaling (FSS) 
exponent, which is mathematically important and practically useful in analyzing finite systems. 
Using the FSS theory of nonequilibrium absorbing phase transitions, we show that the density 
of unsatisfied clauses clearly indicates the transition from the solvable (absorbing) phase to the 
unsolvable (active) phase as varying the noise parameter and the density of constraints. Based on 
the solution clustering (percolation-type) argument, we conjecture two possible values of the FSS 
exponent, which are confirmed reasonably well in numerical simulations for 2 < A' < 3. 
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I. INTRODUCTION 

The iC-satisfiability problem (/(T-SAT) is well known as 
nondeterministic polynomial-time (NP) complete when 
i^ > 3. It is the decision problem of whether an instance 
of Boolean variables can be satisfied by variable assign- 
ments. The instance is the conjunction (A) of clauses and 
each clause is the disjunction (V) of K numbers of vari- 
ables (or negations) . Determining the A'-SAT solvability 
within reasonable computational time is one of principal 
unsolved problems in computer science [l|. Moreover, it 
is fundamentally important, connected to many applica- 
tions. Substantial progress of such constraint satisfaction 
problems (CSPs) has been achieved ^^ by either nu- 
merical or analytical techniques. 

Since the pioneering work for critical behaviors in the 
random iC-SAT by Kirkpatrick and Selman [3|, mathe- 
matics and physics communities have paid attention to 
structural phase transitions in the solution space, their 
scaling behaviors, and the exact locations of transition 
points in the thermodynamic limit. An instance of the 
ii'-SAT can be interpreted as a /-iT-spin interacting system 
in statistical physics and its solution as the ground state 
of the Hamiltonian for the corresponding spin system. 
Based on the interpretation, there are many suggestions 
for deeper connection between the criticality in the spin- 
glass theory and the intractability of the NP complete 
problem as well as many conjectures from both fields by 
trial and error regarding computational hardness. 

However, few systematic tests of critical behaviors were 
presented in the context of finite-size scaling (FSS) [3, |j] . 
In particular, discussions about the FSS exponent and 
the transition nature arc rare due to the difficulty in 
finding exact locations of various transitions in the ther- 
modynamic limit using finite systems, except for K = 2 
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where all the transitions occur at the same location as a 
continuous percolation transition of the solution space. 

There are various solving techniques of CSPs available. 
For large unstructured CSPs, one can solve by either 
general-purpose deterministic algorithms, e.g., Davis- 
Putnam-Logcmann-Loveland (DPLL) [31 (or more tai- 
lored messagepassing algorithms such belief and survey 
propagation Q), or stochastic-local-search (SLS) algo- 
rithms that are generally competitive for large and least- 
structured CSPs, e.g., the random if-SAT. In SLS al- 
gorithms, assigned values to variables are successively 
flipped, based on the local information of algorithmic de- 
tails. Starting with the celebrated simulated annealing 
algorithm by Kirkpatrick et al. [^, several focused SLS 
algorithms have been developed: RandomWalkSAT [0], 
WalkSAT [13, focused Metropolis search (FMS) [ll|, and 
average SAT (ASAT) [13. 

All the solving techniques, however, have difficulties in 
approaching a sharp change of the ensemble for random 
CSPs, namely a "phase transition" . Deterministic one, in 
spite of its exactness, suffers from severely limited system 
sizes, while stochastic one is able to deal with much larger 
system sizes but their results are less accurate than the 
former due to fluctuations and some ambiguity caused by 
the limited simulation time [l3|. 

In this paper, we propose a systematic method to ana- 
lyze data obtained from flnite systems, which can resolve 
numerical accuracy issues from the limited system size, 
the method of sampling, and the computational time. It 
is based on the FSS analysis of nonequilibrium absorbing 
phase transitions (APTs) [ij]. We employ it to charac- 
terize critical behaviors of the transition from the solvable 
[(SOL) absorbing] phase to the unsolvable [(UNSOL) ac- 
tive] phase in the random _ftr-SAT, in terms of the density 
of unsatisfied (UNSAT) clauses as an indicator and a so- 
lution as an absorbing state. 

This paper is organized as follows. In Sec. |lll we de- 
scribe the random ii'-SAT and explain how to explore it 
by ASAT heuristic. In Sec. Illli we suggest relevant phys- 
ical quantities, and discuss the main idea of FSS ansatz 
in perspective of nonequilibrium APTs. We also argue 



scaling properties near and at dynamic SOL/UNSOL 
phase transitions, which are numerically confirmed well 
in Sec. lIVI Finally, we conclude this paper in Sec. |V] with 
the summary of the main results and some remarks. 



tic), such that At = 1/M„(t). Here M„(t) is the total 
number of UNSAT clauses at time t. The simulation is 
terminated either if a solution is found or if the given 
instance is not solved yet until the maximal time, Tmax- 



II. RANDOM K-SAT AND ASAT HEURISTIC 



III. FINITE-SIZE SCALING ANSATZ 



The Boolean expression of an instance F in the random 
K-SAT is written as F = [Ci A C2 A • • • A Cm], where 
each clause Ct is given by Q = {yn V yj2 V • • • V j/i^), 
and each value of yij is randomly assigned from the set 
{xi, ^xi,..., xn, ^xat} of 2iV Boolean variables (them- 
selves and their negation). The above conjunctive norm 
form of F can be also expressed as a bipartite network 
(factor graph) form, too. The density of constraints 
(a = M /N) plays the role of a control parameter in 
the random iC-SAT since it can determine the satisfiabil- 
ity Q and the average solving time of algorithms [la] . As 
a increases, it gets harder to find the SAT configuration 
of variables, and eventually the solution does not exist 
for too large a values. At least one threshold, therefore, 
must exist between the SAT and UNSAT phases. 

Usin g t he most recently developed SLS algorithm, 
ASAT [l2|, we systematically show how to find such a 
threshold value from numerical data of finite systems, de- 
noted as Uc in the thermodynamic limit, as well as critical 
exponents. They correspond to the solvability transition 
point and its critical behaviors, very similar to those of 
nonequilibrium APTs. Among lots of algorithms, ASAT 
deserves to be considered the representative case of our 
new FSS analysis because it is not only the most effi- 
cient focused SLS heuristic but also the simplest variant 
of well-known algorithms with the specific value of the 
noise parameter of ASAT, e.g., RandomWalkSAT. 

Since our main interest is the minimal model study of 
the random ii'-SAT, we here present only the results of 
ASAT and its limiting case, RandomWalkSAT, but our 
analysis techniques can be easily applied to any other 
algorithms (partially tested in [la]). For those who are 
interested in the graph coloring problem (Q-COL), Walk- 
COL in [13] would be the best to be tested by the same 
FSS analysis as what we do. It is because WalkCOL is 
the exact adaptation of ASAT in the random Q-COL. 

We explore the random /ST-SAT by ASAT as follows: 
choose a clause at random among the set of UNSAT 
clauses and then randomly try flipping one assigned value 
out of K variables in the chosen clause. The trial flip is 
accepted with certainty unless the total number of UN- 
SAT clauses, Mu increases, or with a probability p (noise 
parameter) if M^ increases. Whether each trial flip is ac- 
cepted or not, time is incremented by At. In general, one 
takes At = 1/A^ (or l/M ) where N is the total number 
of variables and M = aiV, so that a unit time inter- 
val (Monte Carlo step) corresponds to one trial flip per 
variable on average. However, our choice is restricted to 
UNSAT clauses only in order to improve the simulation 
efficiency of ASAT (by definition, it is a focused heuris- 



So far, critical behaviors near the SOL-UNSOL tran- 
sition have been discussed in terms of the fraction of 
solved/successful samples, Ps{a,N), and the FSS expo- 
nent D that determines the FSS width of a continuous 
phase transition as ]e]iV^/'', where e = {a — ac)/ac- This 
is based on the fact that there is some diverging corre- 
lation volume, ^v ^ ]e]~'' (Cv = N in finite systems at 
6 = 0), like the diverging correlation length as ^ ~ ]£]"" 
{S, = L with L — N^/'^ in d-dimensional finite lattices 
at e = 0). One can find the detailed discussion of i? for 
non-regular lattice types, complex networks, in [l8| . 

Our FSS analysis in the random iiT-SAT follows the 
postulate of a diverging dynamical correlation volume, 
^v, at the solvability transition whose physical manifes- 
tation is the presence of dynamical heterogeneities with 
infinitely many solution states. Using the analogy of the 
FSS concept in the static simulations of nonequilibrium 
APTs, we measure two more physical quantities, (besides 
the solved-sample fraction, Pg), playing roles as good and 
independent indicators in SLS algorithmic phase transi- 
tions: the solving time and the density of UNSAT clauses. 

The solving time, r, can be determined in two ways 
from Psia,N,t) for t < T„iax: (1) T^{a,N) = t* when 
Ps{oi,N,t*) ~ 1/2, corresponding to the median value 
of the solution time set. (2) [r(a,A^)], where [•] de- 
notes an average restricted to SOL trials before T,„ax out 
of all trial samples. Since both are well defined in the 
SOL phase (e < 0), they indicate the transition into the 
UNSOL phase (e > 0) for iV > $v as t - ]e]-''ii, like 
the relaxation time in APTs. Incorporating the size de- 
pendence generally yields T{a,N) = N^h{eN^''^), where 
h{x) ^ x""^!! for large x and r ^ N^ at e = with 
z = v\\lv. In the SOL phase, r approaches a constant as 
A^ — > 00, while in the UNSOL phase it grows exponen- 
tially with N . It is noted that we present t^ only. 

The density of UNSAT clauses, p„(= Mu/N), plays a 
role of another good indicator in the solvability transition 
of the random AT-SAT, namely, "active clause" density as 
if the order parameter of APTs. In applying FSS to its 
critical behaviors, one should notice that the true sta- 
tionary state of a finite system is only the SOL state. To 
learn about the UNSOL state from algorithm tests, one 
should investigate the quasistationary state describing 
the statistical properties of UNSOL trials with some ini- 
tial transient, and determine such quasistationary prop- 
erties from averages over UNSOL representatives out of 
a large independent trial set with random initial condi- 
tions. After the initial transient (depending on both a 
and Af), (p„(a,7V,t)) = {pu{a,N,t))^M/{l- P,{a,N,t)}, 
which gets saturated to {pu(,a,N)) for t ^ N^. Here 
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FIG. 1: (Color online) FSS for 2-SAT by AS AT with 
p — 1/2, where logarithmic corrections to scalings are 
found as (a) (p„(iV,f))iV* = /(i/[iV^/ln(iV)]) and (b) 
rH(Q,7V)/[iV^/ln(iV)] = /i(|e|iV^''^) with e = {a - ac)/ac. 
For the convenience, the same symbols and lines are taken to 
the same system sizes in Figs. [2]and[3l 



FIG. 2: (Color online) FSS for 3-SAT by the limiting 
case of ASAT (p = 1), Random WalkS AT, with logarith- 
mic corrections to scalings as (a) {pu{N,t))[N /ln{N)] = 
f{t/[N-'ln{N)]) and (b) T^{a,N)/[N-^ ln{N)] = h{\e\N'/n. 
The same symbols and lines are taken to the same system 
sizes as in Fig. [T] 



(•) corresponds to an average restricted to UNSOL trials 
and (~) to an average of saturated steady values. 

Near the transition for small |e| and large N, the 
survival UNSAT density is written in the FSS form, 
{pu{a,N)) = N-^gieN^/""), where g{x) ~ a;^^ for a; » 1 
and iV ^ ^v 111 the SOL phase, it trivially scales as 
{pu{a,N)) - N-'^, so 5(2;) ^ |.t|^(i+'') for negatively 
large x. The ac value may also be found by examining 
its A^-dependence as {pu{ac, N)) ^ N~^ since in the SOL 
phase, it falls off as N~^, while in the UNSOL phase, it 
approaches an a-dependent value. 

Finally, we explain the dynamic scaling of {pu{N,t)), 
averaged over survival trials at ac with random initial 
configurations, where the time dependence only involves 
the ratio i/^^^ so that {pu{N,t)) - N-^ f{t/N^). It is, 
however, hard to observe the saturated regime for t ^ 
iV^(^v ^ t^^^) as N increases, so it is better to focus on 
the temporal decay regime for t <C iV^, i.e., ^v ^ A^ 
with the largest A^ value one can test, which enables to 
determine both Uc and 5{= 9 /z) at the same time with 
the pretty good accuracy. This is why {pu{N,t)) ^ i~* 
should be first investigated without any assumption of 
the e value: {pu{N,t)) = t~^F{t/N^), where F{x) ^ 
constant for x <^ 1 and F{x) ^ x^ ioi x ^ 1. 



IV. NUMERICAL RESULTS 

We now present scaling properties tested for 2 < A' < 3 
by ASAT with the noise parameter p, where we set 
Tmax = 10^ and test at most 10^ (5 x 10^) samples for 
2-SAT (3-SAT). The values of p are chosen as follows: 
SLS algorithms may have the optimal p value that ex- 
ists between too less noise to prevent the escape of the 
system from local energy minima and too much fluctua- 
tions. By definition, an optimized algorithm (p = Popt) 
finds solutions with the fastest solving time up to the 
largest a value. It was reported that ASAT for 3-SAT, 
Popt — 0.21, allowing to find solutions up to aim — 4.21 
where the number of flipping variables is linearly propor- 
tional to A^ until a solution is found [I4I. In contrast, 
there are no optimal p values for 2-SAT. It seems to be 
because all the transitions occur at ac = 1 as the mean- 
field (MF) percolation transition with D = 3. Such a 
conjecture has been confirmed by the same FSS test [ig| 
in (2 + A)-SAT with A e [0, 1] (well-discussed in Q). Up 
to a specific X* value (A* = 2/5), it behaves as if 2-SAT 
without the complexity issue of the solution space. Here 
X is the probability for 3-SAT clauses in an instance. 

Figure [U shows FSS tests for 2-SAT by ASAT with 
p = 1/2, where critical exponents are obtained from t 
and pu as varying a, N, and t. In particular, we indi- 
cate the precise ac location as ac = 1.00(2) with 5 — l.Q 




TABLE I: SOL-UNSOL threshold and critical exponents for 
2-SAT and 3-SAT by ASAT with the noise parameter p. 
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FIG. 3: (Color online) FSS for 3-SAT by the optimized 
ASAT (p = popt — 0.21) with logarithmic corrections to 
scalings as (a) (/9„(iV,i))7V® = /(i/[iV71n(iV)]) and (b) 
r^{a,N)/[Nyin{N)] = h{\e\N^^''). The same symbols and 
lines are taken to the same system sizes as in Fig. [1] 



using the plateau and inflection-point analysis of effec- 
tive exponent plots for various system sizes (not shown 
here). Through the conventional FSS analysis, we obtain 
6 = z = 1.0 and i> ~ 3.0, where S ~ 6/z is also checked 
within error bars. Note that logarithmic eorreetion to 
scalings exist as r^ ^ A^^/ln(A^), stemming from the 
presence of quenched disorder in finite CSPs. Scaling be- 
haviors of 2-SAT, including (2-t-X)-SAT for X < X*, do 
not depend on p in ASAT, indeed, and even in the limit- 
ing case of ASAT, RandomWalkSAT (p = 1), as well. In 
spite of the well-known results of 2-SAT, its detailed seal- 
ing properties have rarely been checked systematically for 
finite systems. Thus, our FSS analysis in 2-SAT could be 
a prototype of further applications, including our test in 
3-SAT where we find an interesting result that critical 
behaviors in RandomWalkSAT are quite different from 
those in the optimized ASAT with p = popt = 0.21 using 
the same analysis as Fig. [TJ 

Figure [2] shows FSS tests for 3-SAT by RandomWalk- 
SAT at ac = 2.670(5) with 9 = z = 0.50 and D = 2.0, 
where the precise location of ac is first identified with 
6 = 1.0. These results are exactly the same as those 
in the MF directed percolation (DP) transition with in- 
finitely many absorbing states |14l | , within the SAT phase 
of 3-SAT, even though there are logarithmic corrections 
to scalings again: (p„(t)) -^ [Ht)]°-^yt, {pu{N)) - 
ln{N)/^/N, and t^ ^ VN\n{N), respectively. 

However, the optimized ASAT for 3-SAT (popt = 0.21) 
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exhibits totally different scaling behaviors from those in 
RandomWalkSAT. It is because its transition is located 
well below the clustering and condensation transition 
threshold, ac < ad, where ad — 3.86 in p (related to 
the MF percolation transition of solutions for 3-SAT), 
while that of the optimized ASAT is much above ad and 
rather close to a^ (the SAT-UNSAT threshold). 

At the first sight of Fig. El^a), the FSS collapse of 
{pu{t,N)) does not seems to be good with 9 = z = 1.0 
due to iV-independcnt initial transient before the true 
sealing regime. We find a transition occurring at ac = 
4.185(5) and S = 1.0 accompanying with logarithmic cor- 
rections to scalings. After the initial transient, the true 
scaling regime becomes clearly extended as N increases, 
while in small system sizes, O{10^), such a regime is ab- 
sent. Figure [3{b) shows that D = 3.0 and z = 1.0 with 
logarithmic corrections to scalings (same as Fig. [T]). We 
are also aware at a* = 4.26 (very near as) of a nontrivial 
power-law decay exponent, S ~ 0.20(5) (or logarithmic 
scaling) |16| . but it is not relevant to our current work, 
so not shown here. Relevant numerical results are sum- 
marized in Table [J 



V. SUMMARY 

We have analyzed the random i^-SAT by the simplest 
SLS heuristic in the numerical framework of nonequilib- 
rium APTs. Two possible values of the FSS exponent (D) 
in 3-SAT are conjectured: one is i? = 2 in the directed per- 
colation university class with infinitely many absorbing 
states [14| if ac < ad. The other is /> = 3 in the perco- 
lation university class (same as 2-SAT) if ad < ac < as, 
where ad is the condensation and clustering threshold 
and as is the SAT-UNSAT threshold [1,11], which are nu- 
merically confirmed with logarithmic corrections to scal- 
ings. 

In conclusion, we have a few remarks for further stud- 
ies: dealing with numerical data in .ftT-SAT, one should 
know serious finite-size effects of small systems, N < 
0(10"^). The FSS analysis we tested here would be widely 
applicable to test constraint satisfaction problem (CSP) 
algorithms' performance. The validity check of our re- 
sults and methods could be possible in the graph Q- 
coloring problem (Q-COL). Finally, we suggest that it 
would be interesting to investigate how the sampling bias 
of SLS algorithms discussed in [1^ affects our results (al- 
ready smeared) in universality perspective of the SOL- 
UNSOL transition. 
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